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Abstract 

The  ability  to  simulate  walking  around  in  the  environment  is  a key  element  missing  from  most  of  today's  joint  forces 
simulations.  A number  of  sensor-based  techniques  are  widely  used  to  maneuver  through  Virtual  Environments  but  they 
introduce  artifacts  into  the  interaction.  Mechanical  motion  platforms  have  also  been  applied  to  surmount  these 
difficulties,  but  they  tend  to  exhibit  different  but  equally  troublesome  side  effects  of  their  own.  This  paper  examines  the 
interrelationships  between  virtual  motion  control  and  other  critical  actions  soldiers  need  to  perform  in  VE.  The  goal  is  to 
allow  the  user  to  maneuver  through  VE  in  as  similar  a manner  as  possible  to  walking  through  the  real  world.  If  the 
interactions  between  different  controls  and  sensory  feedback  can  be  made  comparable  to  the  interaction  between  actions 
in  the  real  world,  then  there  is  hope  for  constructing  an  effective  new'  technique.  Human  performance  requirements  are 
viewed  from  an  analytical  standpoint:  pointing  out  the  interactions  between  a full  set  of  virtual  controls  that  would  allow 
the  user  to  act,  sense,  and  react  to  their  environment.  Candidate  solutions  are  discussed  as  the  analysis  is  developed. 
This  has  lead  us  to  a promising  new  design  for  sensor-based  virtual  locomotion  called  Gaiter,  introduced  in  this  paper. 

THE  TECHNOLOGY  AND  ITS  USE 

Virtual  locomotion  is  a control  technique  for  allowing  a person  to  move  naturally  over  long  distances  in  the  Virtual 
Environment  (VE),  while  remaining  within  a relatively  small  physical  space.  Ideally  such  an  interaction  technique  will 
allow  users  to  perform  the  same  coordinated  actions  and  move  with  the  same  agility  in  VE  as  they  do  in  the  real  world. 
Virtual  locomotion  is  essential  for  constructing  Individual  Combatant  Simulators  w’here  the  soldier  interacts  directly  with 
the  surrounding  environment.  Previous  military  simulators  have  focused  on  vehicular  weapons  platforms  where  motion 
was  controlled  through  the  same  kinds  of  control  devices  that  are  used  to  pilot  the  actual  vehicle.  Either  achial  or 
reduced  cost  versions  of  the  controls  used  to  pilot  aircraft  or  drive  a tank  can  be  used  in  flight  and  tank  simulators.  Tins 
made  it  relatively  straightforward  to  attain  a high  level  of  realistic  interaction.  For  a locomotion  simulator,  the  user's 
physical  body  is  the  control  device  and  the  user’s  avatar  (representation  of  the  user's  body  in  VE)  is  the  "vehicle"  being 
controlled.  This  changes  things  considerably,  calling  for  new  kinds  of  control  actions. 

DoD  needs  individual  combatant  simulators  for  both  training  and  analysis.  Such  simulators  can  be  used  to  train  soldiers 
in  team  operations  and  tactical  doctrine.  They  can  be  used  to  familiarize  a group  with  a specific  environment,  and  to  plan 
and  rehearse  operations  in  that  environment.  An  accurate  VE  model  of  an  embassy  could  be  used  to  train  a team  to  carry 
out  an  embassy  evacuation.  For  analysis,  it  is  important  to  adequately  simulate  the  performance  of  individual  combatants 
in  combined  force  simulations  to  make  the  simulation  more  complete  and  enhance  its  validity.  The  individual  combatant 
is  the  "missing  element"  in  today's  military  simulators.  When  this  element  becomes  available,  combined  force  simula- 
tions can  be  used  to  analyze  and  evaluate  new  tactics  and  operational  plans.  It  will  also  allow  new  technology  (for  foot- 
soldiers,  or  jeopardized  by  foot-soldiers)  to  be  studied  with  a comprehensive  simulation. 

HUMAN  FACTORS  ISSUES  INVOLVED 

A Framework  For  Analysis 

The  goal  is  to  find  a virtual  locomotion  control  technique  as  similar  to  actual  locomotion  as  possible.  But  exactly  how 
should  it  be  similar?  We  approach  this  question  through  a careful  analysis  of  the  interrelationship  between  natural  and 
control  actions. 

Specifying  A Control  Technique 

It  is  useful  to  divide  a control  technique  into  two  parts:  the  control  action  made  by  the  user  and  the  controlled  effect 
produced  by  the  computer  system.  Since  in  this  case  the  control  technique  is  used  to  mimic  an  action  performed  in  the 
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real  world,  there  is  a corresponding  natural  action  and  natural  effect.  For  virtual  locomotion  the  natural  action  is 
perambulation  (walking,  running,  sidestepping,  etc.),  the  natural  effect  is  real  motion  (physical  displacement),  and  the 
controlled  effect  is  virtual  motion.  There  are  many  alternative  control  actions  that  can  be  used  for  virtual  locomotion. 
They  will  be  considered  in  terms  of  the  analytical  framework  developed  in  this  section. 

Interaction  With  Other  Natural  Actions 

Natural  locomotion  interacts  with  a wide  variety  of  other  actions.  These  actions  include  looking  (directing  both  the  head 
and  eyes),  manipulation  (typically  involving  the  hands,  as  in  pointing  a rifle),  posturing  (moving  parts  of  the  body,  for 
purposes  other  than  looking,  manipulation,  or  locomotion),  and  thinking.  Thinking  is  included  because  it  is  something 
the  user  does  which  interacts  with  other  actions,  including  locomotion.  Often  the  expression  "cognitive  load"  is  used  to 
describe  how  thinking  can  limit  or  be  limited  by  the  performance  of  other  actions. 

Components  Of  Locomotion 

Locomotion  control  will  be  divided  into  two  components;  control  over  the  direction  of  motion  (steering)  and  the  rate  of 
motion  (speed).  Steering  may  include  a facility  for  sidestepping  (i.e.,  moving  in  a direction  other  than  where  the  torso  is 
oriented). 

Simulating  A Natural  Capability 

The  goal  of  simulated  locomotion  is  to  provide  a control  to  maneuver  through  VEs  as  naturally  as  possible.  We  now' 
have  the  terms  to  state  this  requirement  a little  more  formally.  (1)  The  control  action  should  be  similar  to  the  natural 
action  with  respect  to  their  intrinsic  properties  (constraints  on  movements,  caloric  energy  expenditure,  etc.).  (2)  The 
control  action  should  interact  with  other  actions  (looking,  manipulation,  posturing,  and  thinking)  in  a similar  manner  to 
the  way  the  natural  locomotion  interacts  with  other  actions  (e.g.,  freeing  the  hands  for  manipulation  and  allowing  a 
normal  level  of  coordinated  head,  arm,  and  leg  movements).  (3)  The  components  of  the  control  action  (steering  and 
speed  control)  should  interact  with  each  other  as  the  components  of  the  natural  locomotion  interact.  (4)  The  controlled 
effect  should  be  similar  to  the  natural  effect  (i.e.,  the  rate  and  precision  of  movement  should  match).  This  is  clearly  a tall 
order  for  any  virtual  locomotion  control  to  meet,  but  it  provides  a basic  set  of  criteria  which  can  and  should  be  applied  to 
all  candidate  techniques. 

Compatibility  With  Turning  The  Body 

A great  deal  of  compatibility  with  other  natural  actions  can  be  achieved  by  having  the  user  turn  in  VE  by  physically 
turning  the  body.  This  allows  actions  like  turning  the  head,  pointing  the  hand,  or  wielding  a weapon  to  be  performed  as  a 
natural  coordinated  motion.  It  also  allows  the  reflexive  orientation  a person  might  have  towards  or  away  from  a stimulus 
to  be  used  effectively  (e.g.,  turning  one's  head  to  view  the  source  of  an  unexpected  sound).  Another  advantage  of  using 
the  natural  action  of  turning  is  to  avoid  a potential  sensory  conflict.  People  sense  turning  through  their  visual,  aural, 
haptic,  and  vestibular  systems.  They  build  up  a sense  of  their  orientation  in  an  environment  through  the  temporal 
integration  of  this  information.  Peoples'  vestibular  sense  of  motion  contributes  to  their  ability  to  navigate  through  virtual 
environments. 

We  therefore  prefer  to  treat  turning  the  body  as  an  action  separate  from  locomotion,  limiting  locomotion  to  the  control  of 
the  translational  motion  of  the  body  through  space.  Steering  is  still  required  to  control  the  direction  of  this  motion. 
Although  this  eliminates  the  need  for  the  control  action  to  encompass  turning,  it  imposes  the  constraint  of  having  the 
control  action  be  compatible  with  physical  turning.  The  user  should  be  able  to  perform  virtual  locomotion  while  turning, 
and  the  path  traversed  should  vary  accordingly. 

The  body  can  be  turned  in  several  different  ways.  Most  of  the  time,  people  step  to  turn:  by  lifting  and  planting  their  feet 
in  a different  direction  on  each  successive  step,  the  body  turns  to  face  a new  direction.  People  also  pivot  on  their  feet  to 
turn.  Here  the  foot  is  turned  while  it  continues  to  support  a person's  weight.  There  are  one-  and  two-footed  pivots. 
Pivoting  provides  an  means  of  turning  the  body  that  is  more  economical  in  terms  of  both  time  and  space  than  stepping  to 
turn,  particularly  critical  in  martial  situations. 


Compatibility  With  Real  Motion 
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It  is  highly  desirable  for  the  virtual  locomotion  control  to  be  compatible  with  other  sorts  of  postural  movements  like 
bending  at  the  waist  or  crouching  down.  These  may  be  thought  of  as  special  purpose  actions  - essential  for  certain  tasks. 
A user  might  bend  to  look  around  a comer,  or  crouch  to  hide  behind  an  embankment.  It  would  be  desirable  if  those 
motions  could  be  made  in  a natural  manner  while  performing  virtual  locomotion.  We  do  not  want  users  to  learn  an 
entirely  new  vocabulary  of  control  actions  just  to  allow  them  to  move  in  different  postures. 

It  is  also  desirable  to  allow  virtual  locomotion  to  be  intermixed  with  natural  locomotion.  The  user  could  move  forward  in 
VE  by  making  control  actions  or  by  taking  actual  steps  (albeit  over  a limited  range).  Physical  movements  work  well  in 
VE,  but  must  remain  within  tracker  range  and  within  the  available  space. 

Candidate  Control  Actions 

A variety  of  control  actions  can  be  performed  to  steer  and  set  the  speed  of  motion  through  the  VE.  This  section  will 
describe  sensor  based  controls  that  have  been  employed  for  this  purpose.  The  most  obvious  and  widely  used  steering 
technique  is  that  of  pointing.  With  a vehicular  control,  the  user's  body  remains  fixed  in  the  vehicle  and  a rate  dependent 
pointing  device,  like  a steering  wheel,  is  used  to  turn  the  vehicle.  In  an  individual  combatant  simulator,  the  user  can  turn 
the  body,  so  pointing  can  be  direct. 

Steering  Controls 

Head-based  steering  is  probably  the  most  widely  used  steering  control  for  VE  systems.  It  is  economical  because  the 
same  position  tracker  used  to  determine  the  user's  field-of-view  is  used  to  control  the  direction  of  motion.  Other 
advantages  of  head-based  steering  arc  that  it  "encourages"  the  user  to  look  where  s/he  is  going,  and  the  hands  are  free  for 
manipulation.  The  disadvantage  is  that  looking  and  moving  are  no  longer  independent.  The  user  cannot  turn  the  head  to 
look  to  the  side  while  moving  without  altering  the  path.  Bowman  demonstrated  this  problem  by  asking  users  to  walk  to  a 
position  pointed  at  by  an  arrow  in  VE  [Bowman,  Roller,  & Hodges,  1997],  Users  had  to  keep  turning  to  look  at  the 
arrow  as  they  moved  toward  the  desired  position:  the  act  of  looking  interfered  with  their  motion.  The  task  was  much 
easier  to  perform  using  hand-based  steering. 

Hand-based  steering  techniques  determine  direction  either  from  where  the  arm  is  pointed,  where  a hand-grip  is  pointed, 
or  where  a finger  is  pointed  when  an  instrumented  data-glovc  is  worn.  Hand-based  steering  frees  the  head  for  looking 
and  allows  the  user  to  move  sideways  relative  to  the  head  or  body.  The  disadvantage  is  that  the  control  interferes  with 
manipulation.  The  hand  is  occupied  so  that  using  the  hand  for  other  tasks  leads  to  conflicts  and  interruptions.  The  user 
also  must  remain  aware  of  the  significance  of  where  the  hand  is  pointed. 

Torso-based  steering  frees  the  head  for  looking  and  the  hands  for  manipulation,  but  it  does  not  support  sidestepping. 
Often  people  move  in  the  direction  where  the  front  of  their  torso  is  pointing,  but  sometimes  they  do  not.  A soldier 
aiming  a rifle  across  his  chest  may  prefer  to  advance  in  the  direction  he  is  aiming. 

There  are  three  approaches  to  lean-based  steering:  tilting  the  upper  torso  (bowing),  shifting  weight  relative  to  one’s  feet, 
and  shifting  one’s  weight  relative  to  a platform.  All  three  kinds  of  leaning  provide  hands-free  operation  and  can  support 
sidestepping.  Fairchild,  Tee,  Loo.  Ng,  and  Serra,  [1993]  implemented  a system  based  on  shifting  the  user's  head  position 
by  tilting  the  upper  torso  (bowing).  This  control  is  incompatible  with  the  user  tilting  the  torso  for  other  purposes.  The 
second  approach,  shifting  weight  relative  to  the  feet  (i.e.,  to  the  basis  of  support)  is  of  limited  use  because  the  continuity 
is  broken  when  a user  steps  to  turn. 

The  third  type  of  lean-based  control  is  shifting  one's  weight  relative  to  a platform.  Max  Wells  at  the  University  of 
Washington's  HIT  Lab  has  developed  such  a system  [Wells,  Peterson,  and  Aten,  1996].  Motion  is  controlled  by  moving 
one's  body  locally,  relative  to  a central  neutral  position.  When  immersed  wearing  a head-mounted  display,  the  user 
might  lose  track  of  where  s/he  stands  with  respect  to  the  neutral  position.  The  direction  and  rate  of  optical  flow  provides 
one  indication  of  where  the  user  is  situated.  A set  of  elastic  straps  attached  to  a ring  around  the  user's  waist  gives  haptic 
feedback,  pushing  the  user  back  towards  the  neutral  position.  The  directional  coordinate  frame  is  relative  to  a fixed 
external  point  in  space,  an  unnatural  condition  that  produces  interesting  side  effects.  This  makes  turning  the  body  and 
controlling  the  direction  of  motion  even  more  independent  than  they  are  with  natural  locomotion.  The  user  may  choose 
to  move  in  one  direction  and  then  turn  to  face  in  another,  making  it  as  easy  to  "run"  backwards  as  forward.  This 
approach  is  incompatible  with  physical  locomotion  because  velocity  is  controlled  by  relative  position  of  the  body. 


8-4 


Speed  Controls 

Speed  can  be  controlled  using  finger  pressure,  by  the  degree  of  leaning  (when  leaning  is  used  for  steering),  by  the  rate  at 
which  the  user  steps  in-place,  or  by  the  degree  of  leg  movement  when  the  user  steps  m-place.  Only  the  use  of  finger 
pressure  will  be  discussed  in  this  section.  Often  a pair  of  binary  switches  attached  to  a hand  control  are  used  to  invoke 
either  forward  or  backward  virtual  motion.  This  is  a widely  used  technique,  easy  and  inexpensive  to  implement.  We 
have  tried  using  pressure  sensitive  buttons  and  recommend  their  use  because  it  provides  a smooth  way  of  varying  speed. 
A major  advantage  of  hand  controls  is  that  they  work  independently  of  head,  torso,  and  leg  movement  and  are  thus 
compatible  with  a wide  range  of  physical  motions.  The  primary  drawback  of  using  the  hand  to  control  speed  is  that  it 
interferes  with  using  the  fingers  for  manipulative  tasks.  As  VEs  become  richer  and  engage  the  user  in  more  complex 
tasks,  they  require  a greater  commitment  of  the  user’s  hands  for  manipulation.  This  is  most  evident  for  combat  systems 
where  the  user  needs  to  hold  and  aim  a weapon  both  at  rest  and  in  motion.  A second  drawback  of  using  hand  controls 
with  head-mounted  displays  is  that  the  user  cannot  directly  see  how  his  hand  holds  the  grip  or  how  fingers  touch  the 
buttons.  This  limits  the  number  of  buttons  the  user  can  deal  with. 

REVIEW  OF  THE  STATE-OF-THE-ART 

This  review  will  cover  both  mechanical  motion  platforms  and  a promising  new  sensor-based  virtual  locomotion  system. 

Mechanical  Locomotion  Systems 
Uni-directional  Systems 

A number  of  mechanical  solutions  have  been  proposed.  One  class  of  device  is  the  uni-directional  systems  that  limit 
movement  to  one  direction  and  require  a special  control  action  for  turning  the  virtual  world  around  the  user.  An  early 
system  is  the  treadmill  used  at  University  of  North  Carolina  in  their  Architectural  Walkthrough  System  [Brooks,  Airey, 
Alspaugh,  Bell,  Brown,  Nimscheck,  Rheingans,  Rohlf,  Smith,  Turner,  Varshney,  Wang,  Wber,  & Yuan,  1992].  It  uses  a 
handle  in  front  that  the  user  twists  to  steer.  They  report  that  the  treadmill  is  disorienting,  probably  because  there  is  no 
true  sense  of  turning  which  is  helpful  for  route  learning.  Spinning  the  virtual  world  about  the  user  produces  an  odd 
transient  effect.  Other  uni-direction  systems  possibly  suffer  from  the  same  disadvantage.  Sarcos  Research  Corporation 
developed  the  TREADPORT,  a large  treadmill  that  allows  the  user  to  stand,  kneel,  sit,  and  lay  prone  [Youngblut,  1996], 
The  user  is  tethered  for  safety.  OSIRIS,  developed  by  the  US  Army's  Night  Vision  Laboratory,  is  a stair-stepper  device 
that  can  only  be  used  upright  [Lorenzo  et  al.,  1995],  It  can  present  forward  motion  only;  a hand  control  is  used  for 
steering.  The  Individual  Soldier  Mobility  Simulator  (ISMS)  was  developed  by  Sarcos  Research  Corporation  in  conjunc- 
tion with  the  Army  Research  Institute.  The  user  stands  on  two  robotic  arm  boot  plates  that  create  the  sense  of  walking  or 
running  over  different  terrain,  and  ascending  or  descending  stairs  [Lorenzo  et  al.,  1995], 

Multi-directional  Systems 

Other  mechanical  systems  allow  the  user  to  move  in  multiple  directions.  Sarcos  Research  Corporation's  Uniport  is  based 
on  a unicycle  [Lorenzo,  Poole,  Deaso,  Lu,  Kekesi,  Cha,  Slayton,  Williams,  Moulton,  Kaste,  MacKrell,  Paddison,  Rieks, 
Roth,  & Wodoslawsky,  1995],  The  user  remains  in  a seated  position  and  turns  around  a pivot  point  by  applying  torque  to 
the  seat.  This  engages  a motor  to  turn  the  seal  at  a controlled  rate.  (This  system  is  at  the  borderline  between  a vehicular 
and  locomotion  control.)  The  user's  hands  are  free  to  hold  a gun  or  other  objects. 

The  Virtual  Perambulator  [Iwata  and  Fujii,  1996]  uses  a sliding  motion  of  the  feet  to  indicate  walking.  The  user  wears 
sandals  with  low  friction  film  on  the  middle  of  the  sole  and  a rubber  brake  pad  at  the  toe.  The  user  glides  on  a low 
friction  surface  by  pushing  the  waist  against  a hoop  that  surrounds  the  user  and  sliding  the  feet.  A position  sensor 
attached  to  each  ankle  and  contact  sensors  on  the  bottom  of  each  foot  allows  the  system  to  recognize  the  length  and 
direction  of  each  step  to  specify  the  movement  in  VE.  An  earlier  version  used  roller  skates  instead  of  the  low  friction 
film.  The  placement  of  the  hoop  at  waist  level  does  not  allow  a user  to  hold  an  object  by  the  side. 

The  Omni-Directional  Treadmill  (ODT)  developed  by  David  Carmein  of  Virtual  Space  Devices  Inc.  [Darken,  Cockayne, 
& Carmein  1997]  allows  the  user  to  walk  in  any  direction.  The  mechanism  consists  of  a pair  of  conveyor  belts  nested 
one  inside  the  other.  Each  track  moves  horizontally  and  the  tracks  are  oriented  perpendicularly  to  each  other.  The  outer 
track  has  rollers  on  it  so  that  it  can  transmit  the  motion  produced  by  the  inner  track.  The  rollers  can  thus  convey  motion 
in  any  direction  to  the  user's  feet  resting  upon  it.  The  active  surface  of  the  motion  platform  measures  50"x  50".  A 
control  system  is  used  that  seeks  to  keep  the  user  centered  in  the  middle  of  the  platform.  The  system  is  very  effective  at 
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allowing  a user  to  walk  in  a straight  line  in  any  direction.  People  have  little  difficulty  accelerating  along  a straight  path. 
Turning  while  accelerating  can  lead  to  a misalignment  between  the  user’s  direction  of  translation  and  the  centering 
motion  of  the  mechanism.  This  is  apt  to  make  the  user  lose  his/her  balance  and  stumble.  Even  turning  in-place  can  be 
difficult  because  the  tracking  system  registers  the  motion  and  compensates  for  it  by  moving  the  surface  under  the  user's 
feet.  Sometimes  the  dynamics  of  natural  locomotion  go  beyond  what  the  ODT  can  support.  People  can  decelerate  very 
quickly,  coming  to  a full  stop  in  a single  step,  and  if  they  pivot  while  walking  fast  they  can  redirect  their  motion  even 
more  rapidly.  Further,  people  perceive  linear  and  angular  acceleration  using  their  vestibular  system.  A user  immersed  in 
VE,  standing  on  a "perfect"  motion  platform  would  have  the  sensation  that  the  vestibular  perception  of  linear  acceleration 
were  "turned  off".  This  alone  will  make  it  difficult  to  perform  certain  actions,  even  if  the  dynamic  response  of  a motion 
platform  were  perfect. 

First  Attempt  Using  Walking-In-Place 

The  first  virtual  locomotion  system  we  implemented  and  tested  used  a combination  of  torso-based  steering  and  step- 
based  speed  control.  Sidestepping  was  not  supported.  A position  tracker  was  attached  to  the  user's  waist  and  pressure 
sensors  were  mounted  under  the  ball  and  heel  of  each  foot  on  the  insoles  of  their  shoes.  The  basic  idea  was  to  initiate 
virtual  motion  when  the  user  took  an  in-place  step  and  use  the  rate  of  stepping  to  control  speed,  The  system  also  had  to 
be  able  to  distinguish  between  stepping  to  turn  and  stepping  in-place  to  initiate  virtual  motion.  We  made  the  mistake  of 
putting  the  burden  of  making  this  distinction  on  the  user.  Forward  and  backward  virtual  motion  were  indicated  by 
pushing  off  the  ball  or  heel  of  the  foot,  respectively.  Flat  footed  steps  produced  no  virtual  displacement  and  were  to  be 
used  for  stepping  to  turn.  The  user  could  also  turn  while  advancing  in  VE  by  pushing  off  the  ball  of  their  foot  while 
stepping  to  turn  in-place.  This  was  easy  to  do  when  the  user  concentrated  on  it,  and  the  computer  had  no  problem  rec- 
ognizing these  distinctions.  But  users  tend  to  get  a little  "sloppy"  when  they  started  thinking  about  something  other  than 
how  they  stepped.  Although  ninety  percent  of  their  steps  indicated  their  intention,  the  remainder  is  enough  to  spoil  the 
technique.  Unintentionally  movement,  or  missing  a step  is  quite  disconcerting. 

Gaiter:  Our  New  Locomotion  Control 

We  developed  a second  generation  locomotion  control  to  allow  the  user  to  more  naturally  control  their  movements 
(without  having  to  concentrate  on  their  footsteps)  and  to  support  sidestepping.  We  call  the  new  system  Gaiter.  Its  design 
is  based  on  generalized  classification  of  human  gait  that  encompasses  sidestepping  and  in-place  stepping  as  well  as 
forward  stepping. 

The  user  walks  in-place  to  walk  through  VE.  The  user  can  move  in  any  direction  by  rocking  the  knees.  Forward  virtual 
motion  is  achieved  by  simply  walking  in  place.  In  this  case  the  knees  rock  forward  then  back.  To  take  a virtual  sidestep 
to  the  right  the  user  lifts  his  right  knee  up  to  the  right  and  then  drops  it  back  down  (to  the  left);  the  left  foot  should  be 
raised  and  lowered  on  alternate  steps,  but  need  not  rock  horizontally.  Backward  virtual  steps  are  taken  by  rocking  the 
knees  backward  first  then  forward.  The  user  can  rock  the  knees  along  a diagonal  path  to  move  diagonally  through  VE. 
The  direction  and  degree  of  virtual  motion  is  determined  by  direction  and  degree  with  which  the  knee  moves.  Virtual 
displacement  is  a function  of  both  how  far  the  knees  swing  and  the  stepping  rate,  as  physical  displacement  is  a function 
of  both  stride  length  and  stepping  rate. 

Gaiter  employs  a hybrid  sensor  system.  Position  trackers  placed  on  the  knees  register  knee  motion.  Pressure  sensors 
placed  on  shoe  insoles  under  the  ball  and  heel  of  each  foot  register  ground  reaction  forces.  The  pressure  sensors  help 
determine  the  timing  of  each  step,  punctuating  knee  motion.  Pattern  recognition  software  distinguishes  between  steps  to 
turn,  in-place  steps,  and  actual  steps.  Since  the  system  can  distinguish  between  them,  it  can  allow  the  user  to  intermix 
virtual  and  actual  steps.  Virtual  steps  move  the  user  in  the  VE  while  remaining  stationary  in  the  real  world. 
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Gaiter:  a new  virtual  locomotion  control  being  developed  at  NRL.  The  user  walks  in-place  to  walk  through  VE. 
Steps  are  characterized  using  a combination  of  position  trackers  and  pressure  sensors 

Actual  steps  move  the  user  the  same  distance  in  both  the  virtual  and  real  world.  Actual  steps  provide  the  most  natural 
and  accurately  judged  movements  but  their  range  is  limited,  so  virtual  steps  are  used  to  cover  large  distances. 

Features  Of  Gaiter 

Many  of  the  "realistic"  features  of  Gaiter  derive  from  its  adoption  of  leg  motion  as  the  control  action.  It  operates  in  the 
appropriate  coordinate  frame  of  reference:  the  direction  of  knee  movement  with  respect  to  the  surrounding  environment. 
Virtual  motion  may  be  made  in  any  direction.  Sidestepping  is  an  inherent  part  of  the  technique.  It  is  compatible  with 
turning  the  head  and  body,  using  the  hands  for  other  tasks,  a variety  of  postural  movements,  and  actual  walking.  The 
system  can  be  tuned  to  preserve  metrics  between  physical  and  virtual  space.  Knee  movement  can  be  scaled  and  mapped 
into  virtual  displacement.  The  system  is  responsive;  the  pattern  recognition  system  does  not  have  to  wait  for  the  gesture 
to  be  complete  before  responding  to  knee  movements.  Virtual  motion  occurs  as  the  knees  rock.  The  equipment  used  to 
implement  Gaiter  is  compact  and  relatively  inexpensive.  A wireless  version  can  be  built  today,  albeit  at  greater  expense 
than  using  tethered  sensors. 

The  basic  limitation  of  Gaiter  is  that  walking  and  running  in-place  are  not  the  same  as  actual  walking  and  mnning.  They 
use  somewhat  different  muscle  sets  in  somewhat  different  ways.  The  user  does  not  physically  accelerate  along  the  path 
of  motion  or  build  up  momentum  by  mnning  faster  in  place.  (It  may  be  possible  to  simulate  the  latter  effect  by 
increasing  virtual  velocity  as  successive  strides  are  made,  but  this  may  make  the  requisite  deceleration  seem  more  artifi- 
cial.) It  will  be  possible  to  tune  the  system  to  match  particular  attributes  of  natural  locomotion  (e.g.,  perceived  velocity, 
natural  cadence,  caloric  expenditure,  etc.)  but  it  is  unlikely  that  one  set  of  tuning  parameters  will  satisfy  all  criteria. 

Experimental  Evaluation 

We  are  currently  mnning  pilot  studies  comparing  three  different  sensor-based  virtual  locomotion  controls:  head-based, 
hand-based,  and  leg-based  (Gaiter).  We  have  started  by  using  the  set  ofloeomotion  tasks  described  as  part  of  VF.PAB 
(ARI's  VE  Performance  Assessment  Battery  [Lampton,  Knerr,  Goldberg,  Bliss,  Moshell,  & Blau,  1994]).  We  also  have 
models  of  the  Quantico  Village  database  developed  by  Paradigm,  Inc.  for  the  Team  Tactical  Engagement  System  for 
NAWC/TSD  and  the  Ex-USS  Shadwell  ship  interior  developed  Dave  Tate  at  NRL  [Tate,  Sibert,  & King,  1997],  This 
gives  us  a variety  of  interesting  virtual  spaces  to  walk  through.  We  are  tuning  the  current  version  of  Gaiter  to  make  it 
match  peoples'  natural  strides.  The  tasks  we  will  use  in  the  comparative  experiment  range  from  simple  locomotion  down 
hallways  and  through  rooms,  to  the  more  complex  task  of  clearing  a building  by  pointing  and  clicking  at  targets  while 
minimizing  one's  exposure  to  them.  Our  hypothesis  is  that  people  will  perform  tasks  differently  using  the  three  different 
controls  and  the  sequence  of  actions  (including  looking,  moving,  and  manipulation)  they  perform  using  the  leg-based 
control  will  best  match  the  sequence  of  actions  used  to  perform  the  task  in  the  real  world. 
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LIMITATIONS  OF  THE  TECHNOLOGY 

There  are  many  limitations  to  the  sensors,  processing  systems,  and  displays  used  to  construct  today's  VE  systems. 
Spatial  position  trackers  have  a limited  range,  are  sensitive  to  metal  and  interference  from  other  sources,  and  are  often 
tethered.  Currently  available  head  mounted  displays  light  weight  enough  to  wear  while  walking  offer  a limited  field-of- 
view  (typically  50-degrees),  a limited  display  resolution  (typically  640  x 480  pixels),  a fixed  depth  of  focus,  and  are  often 
tethered.  Current  rendering  systems  provide  limited  realism  of  real-time  imagery.  The  lag  due  to  sensor  transmission, 
image  generation,  and  display  contributes  to  motion  sickness  [DiZio  & Lackner,  1992], 

There  are  clearly  limitations  to  the  physical  interaction  provided  by  VE  systems.  In  terms  of  locomotion  the  ground 
surface  is  almost  always  flat,  with  fixed  surface  characteristics.  This  could  be  improved  by  employing  a hydraulically 
tilted  floor,  with  adjustable  surface  compliance.  The  remaining  omission  would  then  be  the  absence  of  the  collision 
forces  made  contacting  walls  and  other  raised  surfaces.  The  challenge  will  be  to  add  these  features  while  keeping  the 
system  safe  enough  to  use  without  falling. 


Consequences  Of  Unrealistic  Interaction 

What  happens  when  the  requirements  for  interactive  realism  stated  earlier  do  not  hold;  when  similarity  of  (1)  action,  (2) 
interaction,  (3)  component  actions,  or  (4)  effect  are  not  fully  attained?  The  actions  must  be  the  same  for  the  user  to 
develop  the  right  set  of  coordinated  reflexes.  The  capability  (effect)  must  be  the  same  for  users  to  learn  when  to  apply  it, 
to  incorporate  it  into  strategies,  and  to  avoid  changing  their  strategy  to  compensate  for  its  absence.  Compensation  is  bad 
because  it  means  that  users  are  adopting  different  practices  and  approaches  in  VE  than  they  would  use  in  the  real  world. 

In  terms  of  isolated  capabilities,  capabilities  can  be  lost  in  VE,  new  capabilities  can  he  gained,  and  the  control  action  can 
alter  the  way  a capability  is  used.  All  of  these  cases  are  departures  from  realistic  task  performance.  If  a control  lets  the 
user  effortlessly  move  at  a rapid  speed,  the  user  might  carry  out  rehearsal  missions  in  VE  that  would  be  impossible  to 
accomplish  in  the  real  world.  In  terms  of  interacting  capabilities  - the  control  action  can  preclude  other  capabilities  (e.g., 
not  being  able  to  stop  quickly  may  discourage  rapid  motion),  the  control  action  can  free  the  user  to  perform  extra  actions 
(e.g.,  a technique  that  supports  backwards  motion  while  the  user  stands  at  rest  would  allow  the  user  to  aim  a rifle 
accurately  while  retreating  from  the  enemy),  and  the  control  action  can  alter  the  way  other  capabilities  are  used  (e.g.,  not 
being  able  to  advance  in  a crouching  posture  limits  the  kind  of  ground  cover  one  can  hide  behind). 

PROJECTS  USING  THIS  TECHNOLOGY 

Many  applications  would  benefit  from  the  ability  to  walk  through  VE  in  a realistic  manner.  In  the  Team  Tactical 
Engagement  Simulator  (TTES)  developed  by  the  Naval  Air  Warfare  Center  Training  Systems  Division  (NAWC/TSD) 
and  Southwest  Research,  military  personnel  practice  urban  warfare  tactics  [Horey  Fowlkes,  & Reir,  1995;  Goodman, 
Porter,  & Standridge,  1997],  The  Naval  Research  Laboratory's  Information  Technology  Division  developed  a VE  of 
portions  of  the  Ex-USS  Shadwell,  the  Navy's  full-scale  fire  research  and  test  ship,  to  test  the  feasibility  of  using 
immersive  VE  as  a tool  for  shipboard  firefighting  training  and  mission  rehearsal  [Tate,  Sibert,  William,  King,  & Hewitt, 
1995].  Navy  firefighters  who  practiced  in  the  three  dimensional  model  that  included  fire  and  smoke  effects  to  simulate 
actual  firefighting  conditions  performed  better  than  those  who  only  practiced  using  the  ship's  diagrams  [Tate  et  ah,  1997], 
The  University  of  North  Carolina's  Architectural  Walkthrough  System  allows  designers  to  explore  a building  design 
before  its  construction  [Brooks  et  al.,  1992], 

DIRECTIONS  FOR  THE  FUTURE 

We  will  continue  to  refine  Gaiter.  There  are  different  ways  of  mapping  the  characteristics  of  in-place  stepping  to  virtual 
stepping.  We  have  initially  adopted  a linear  mapping  between  the  displacement  of  the  knee  while  stepping  in-place  to 
the  displacement  of  a virtual  step.  Further  studies  will  be  carried  out  to  explore  how  well  this  matches  natural  stride 
lengths  and  stepping  tempo.  A variety  of  criteria  will  be  used  to  measure  the  fit  including  the  user’s  perceived  rate  of 
motion,  the  energy  expended,  or  subjective  effort  made  to  move  at  different  speeds. 
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We  are  developing  an  avatar  driven  by  Gaiter.  The  avatar  has  two  basic  uses.  It  allows  the  user  to  see  how  his  body  fits 
into  the  VE,  giving  the  user  a better  sense  of  immersion  [Slater  Usoh,  & Steed,  1995].  An  avatar  also  lets  other  people 
see  the  user's  pose  and  location  in  VE.  The  challenge  of  developing  an  avatar  driven  by  Gaiter  is  to  match  the  control 
actions  (in-place  stepping)  to  the  controlled  effects  (virtual  stepping)  in  as  natural  a way  as  possible.  The  match  will  not 
be  perfect  and  it  remains  to  be  seen  how  tolerant  users  will  be  to  different  kinds  of  mismatch. 

CONCLUSIONS 

There  are  many  interactions  between  perceptual-motor  systems  and  candidate  locomotion  controls  involving  looking, 
manipulating,  turning  the  body,  posturing,  and  thinking.  These  interactions  limit  the  most  promising  approaches  to  those 
using  the  legs  to  make  actions  similar  to  natural  locomotion.  Our  first  attempt  at  building  a leg-based  control  put  too 
high  a burden  on  the  user  to  make  special  kinds  of  steps.  We  combined  position  trackers  on  the  legs  with  the  insole 
pressure  sensors  to  create  a walking  simulator  called  Gaiter.  Gaiter  provides  a consistent  means  of  taking  virtual  steps  in 
any  direction.  We  will  test  our  new  virtual  locomotion  technique  against  two  common  sensor-based  techniques:  head- 
based  steering  and  hand-based  steering,  both  using  finger  pressure  to  control  speed.  The  controls  will  be  evaluated  in 
terms  of  how  similar  peoples'  actions  in  VE  are  using  the  control  compared  to  peoples'  actions  in  the  real  world. 
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